Technique of Determining a Measure of Proximity between Two Devices

ABSTRACT

Disclosed is a technique of determining a measure of proximity between two devices ( 4, 6 ). A method implementation of the technique comprises obtaining a first device signature comprising an indication of a first point in time and a first parameter characteristic of a first measurement performed by a first sensor ( 10 ) comprised in the first device ( 4 ); obtaining a second device signature comprising an indication of a second point in time and a second parameter characteristic of a second measurement performed by a second sensor ( 12 ) comprised in the second device ( 6 ); and determining, based on the first device signature and the second device signature, the measure of proximity between the first device ( 4 ) and the second device ( 6 ).

TECHNICAL FIELD

The present disclosure generally relates to device tracking. Inparticular, a technique for determining, based on device signatures, ameasure of proximity between a first device that is mobile and a seconddevice is presented. The technique may be implemented in the form of amethod, an apparatus or a computer program product. Further, a devicefor generating a device signature is presented.

BACKGROUND

In several fields, tracking requires the determination of the locationof a device in terms of absolute and relative position. Absolutetracking means that the position of the device is known in some fixedcoordinate space such as Global positioning System, GPS, coordinates ora shelf number. In many cases, for example in logistics applications,there is room for a much coarser position estimate, and it is moreinformative to provide relative instead of absolute positions.

FIG. 1 shows parcels #1 to #3 and different ways of transportationthereof. Some parcels are carried together at some point, then they maybe stored in the same room, then repackaged with other parcels and thenshipped to their final destinations. When describing the position of theparcels in relative terms, it is of interest which parcels are packed inthe same pellet, which truck they are travelling in and which room aparcel is in. A relative position describes the position of a devicerelative to other potentially moving objects, e.g., whether the deviceis in a specific container, in a cargo ship, or remote from anotherdevice.

Conventional device tracking methods use labels attached to devices anda method to read the labels and assign them to certain positions. Forinstance, bar codes or RFID tags can be attached onto devices and thenscanned to assign them to the certain positions. Label-based trackingmethods are tedious and prone to error. In particular, labels may bemisread and assigned to a wrong location. Also, label reading usually isnot highly automated.

Tracking systems based on a Global Navigation Satellite System, GNSS,are frequently used for tracking devices outdoors. GNSS signals cannotpenetrate walls. Therefore, locations of devices which are containedwithin other objects, containers or inside vehicles cannot be reliablydetermined using GNSS.

Radio-based tracking relies on emitting a signal by the device and asubsequent triangulation by multiple base stations. Such positioningsolutions have precision problems—even in an empty space their positionaccuracy is limited, but in a space filled with objects, their precisiondegrades significantly, for example due to fading of the emittedsignals.

SUMMARY

There is a need for a technique that enables the determination of arelative position of a device with respect to another device.

According to a first aspect, a method of determining a measure ofproximity between a first device that is mobile and a second device isprovided. The method comprises obtaining a first device signaturecomprising an indication of a first point in time and a first parameterassociated with the first point in time, wherein the first parameter ischaracteristic of a first measurement performed by a first sensorcomprised in the first device. The method further comprises obtaining asecond device signature comprising an indication of a second point intime and a second parameter associated with the second point in time,wherein the second parameter is characteristic of a second measurementperformed by a second sensor comprised in the second device.Additionally, the method comprises determining, based on the firstdevice signature and the second device signature, the measure ofproximity between the first device and the second device. The method isin one variant a computer-implemented method, i.e., a method executed bya processor.

The measure of proximity can be a binary indication describing that thefirst device and the second device are in a same locally restricted(e.g., contained) environment, such as the same room, a same shippingcontainer, a same parcel or else. The measure of proximity may be avalue indicating a probability with which the first device and thesecond device are in a same room, a same shipping container, a sameparcel or else. The measure of proximity may be a value describing adistance between the first device and the second device or a relativeposition of the first device and the second device.

In one variant, the first measurement consists of a first amount ofinformation and the first parameter consists of an amount of informationlower than the first amount. Additionally, or in the alternative, thesecond measurement may consist of a second amount of information and thesecond parameter may then consist of an amount of information lower thanthe second amount.

The first device signature may further comprise an indication of a thirdpoint in time and a third parameter associated with the third point intime. The third parameter may be characteristic of a third measurementperformed by a third sensor comprised in the first device. According toone variant, the third measurement consists of a third amount ofinformation and the third parameter consists of an amount of informationlower than the third amount.

The second device signature for example further comprises an indicationof a fourth point in time and a fourth parameter associated with thefourth point in time. For example, the fourth parameter ischaracteristic of a fourth measurement performed by a fourth sensorcomprised in the second device. The fourth measurement may consist of afourth amount of information. In this case, the fourth parameter mayconsist of an amount of information lower than the fourth amount.

The determination of the measure of proximity is in one exampleperformed by a neural network.

The determining the measure of proximity may comprise correlating thefirst device signature and the second device signature. The correlatingin one example comprises comparing the first point in time with thesecond point in time. The correlating may comprise comparing the firstparameter with the second parameter.

In one variant, the correlating comprises comparing the third point intime with the fourth point in time. The correlating may comprisecomparing the third parameter with the fourth parameter.

The first signature for example comprises at least one further entrychosen from: an indication of an amount of parameters comprised in thefirst signature; an indication of the sensor type of the first sensor;an indication of a point in time at which the first measurement wasstarted by the first sensor; an indication of a timespan between thepoint in time at which the first measurement was started by the firstsensor and the first point in time; an indication of the parameter typeof the first parameter; an indication of the sensor type of the thirdsensor; an indication of a point in time at which the third measurementwas started by the third sensor; an indication of a timespan between thepoint in time at which the third measurement was started by the thirdsensor and the third point in time; and an indication of the parametertype of the third parameter.

In one variant, the second signature comprises at least one furtherentry chosen from: an indication of an amount of parameters comprised inthe second signature;

an indication of the sensor type of the second sensor; an indication ofa point in time at which the second measurement was started by thesecond sensor; an indication of a timespan between the point in time atwhich the second measurement was started by the second sensor and thesecond point in time; an indication of the parameter type of the secondparameter; an indication of the sensor type of the fourth sensor; anindication of a point in time at which the fourth measurement wasstarted by the fourth sensor; an indication of a timespan between thepoint in time at which the fourth measurement was started by the fourthsensor and the fourth point in time; and an indication of the parametertype of the fourth parameter.

The sensor type for example indicates a physical property which thesensor is configured to measure. The first sensor and the second sensorare for example configured to measure a same physical property. In oneexample, the third sensor and the fourth sensor are configured tomeasure a same physical property. In one variant, the first sensor andthe second sensor are configured to measure a physical propertydifferent from a physical property measured by the third sensor and thefourth sensor. The physical property measured by any sensor type may beone of vibration, sound, light, acceleration, rotation, magnetic fieldor temperature.

The first parameter and the second parameter are in one exampleparameters of a same parameter type. The third parameter and the fourthparameter may be parameters of a same parameter type. For example, theparameter type is one of a maximum value, a minimum value, a value abovea predetermined threshold or a value below a predetermined threshold,wherein the value is a value of the measurement, a derivative of themeasurement with respect to time, a Fourier-transform of themeasurement, a quaternion of the measurement, a Mel spectrum of themeasurement, a histogram of the measurement, or a wavelet-transform ofthe measurement.

In one example, at least one sensor chosen from the first sensor, thesecond sensor, the third sensor and the fourth sensor is a camera. Forexample, the first sensor is a camera. The first parameter may then be avalue of a histogram of the first measurement by the camera. Thehistogram may then represent a color distribution of pixels contained inan image acquired by the camera as the first measurement. For example,the second sensor is a camera. The second parameter may then be a valueof a histogram of the second measurement by the camera. The histogrammay then represent a color distribution of pixels contained in an imageacquired by the camera as the second measurement. For example, the thirdsensor is a camera. The third parameter may then be a value of ahistogram of the third measurement by the camera. The histogram may thenrepresent a color distribution of pixels contained in an image acquiredby the camera as the third measurement. For example, the fourth sensoris a camera. The fourth parameter may then be a value of a histogram ofthe fourth measurement by the camera. The histogram may then represent acolor is distribution of pixels contained in an image acquired by thecamera as the fourth measurement. The image may be in RGB or HSV colorspace.

In one variant, at least one sensor chosen from the first sensor, thesecond sensor, the third sensor and the fourth sensor is a microphone.For example, the first sensor is a microphone. The first parameter maythen be a value of a Fourier-transform of the first measurement or awavelet-transform of the first measurement. For example, the secondsensor is a microphone. The second parameter may then be a value of aFourier-transform of the second measurement or a wavelet-transform ofthe second measurement. For example, the third sensor is a microphone.The third parameter may then be a value of a Fourier-transform of thethird measurement or a wavelet-transform of the third measurement. Forexample, the fourth sensor is a microphone. The fourth parameter maythen be a value of a Fourier-transform of the fourth measurement or awavelet-transform of the fourth measurement.

In case at least one of the first, the second, the third and the thirdsensor is a microphone, the corresponding parameter may be a value ofthe corresponding measurement, the value describing a sudden temporalchange in a spectrum of the corresponding measurement. For example, thefirst sensor is a microphone and the first parameter is a value of thefirst measurement, the measurement describing a sudden temporal changein a spectrum of the first measurement. For example, the second sensoris a microphone and the second parameter is a value of the firstmeasurement, the measurement describing a sudden temporal change in aspectrum of the second measurement. For example, the third sensor is amicrophone and the third parameter is a value of the third measurement,the measurement describing a sudden temporal change in a spectrum of thethird measurement. For example, the fourth sensor is a microphone andthe fourth parameter is a value of the fourth measurement, themeasurement describing a sudden temporal change in a spectrum of thefourth measurement.

In one example, at least one of the sensors is a microphone and thecorresponding parameter is a value of a Mel spectrogram of the firstmeasurement. For example, the first sensor is a microphone and the firstparameter is a value of a Mel spectrogram of the first measurement. Forexample, the second sensor is a microphone and the second parameter is avalue of a Mel spectrogram of the second measurement. For example, thethird sensor is a microphone and the third parameter is a value of a Melspectrogram of the third measurement. For example, the fourth sensor isa microphone and the fourth parameter is a value of a Mel spectrogram ofthe fourth measurement. In each case, the Mel spectrogram may be acompressed spectrogram.

In one example, at least one sensor chosen from the first, the second,the third and the fourth sensor is a gyroscope, an acceleration sensoror an inertial motion sensor. In this case, the corresponding parametermay be a value of the corresponding measurement describing an amount ofrotation of the respective device measured by the sensor. Thecorresponding parameter may in this case be a value of the correspondingmeasurement describing an incremental rotation of the respective devicemeasured by the sensor, or a value of a quaternion of the correspondingmeasurement. For example, the first sensor is a gyroscope, anacceleration sensor or an inertial motion sensor. Then, the firstparameter may be a value of the first measurement describing an amountof rotation of the first device measured by the first sensor. The firstparameter may be a value of the first measurement describing anincremental rotation of the first device measured by the first sensor,or a value of a quaternion of the first measurement. For example, thesecond sensor is a gyroscope, an acceleration sensor or an inertialmotion sensor. Then, the second parameter may be a value of the secondmeasurement describing an amount of rotation of the second devicemeasured by the second sensor. The second parameter may in this case bea value of the second measurement describing an incremental rotation ofthe second device measured by the second sensor, or a value of aquaternion of the second measurement. For example, the third sensor is agyroscope, an acceleration sensor or an inertial motion sensor. Then,the third parameter may be a value of the third measurement describingan amount of rotation of the first device measured by the third sensor.The third parameter may in this case be a value of the third measurementdescribing an incremental rotation of the first device measured by thethird sensor, or a value of a quaternion of the third measurement. Forexample, the fourth sensor is a gyroscope, an acceleration sensor or aninertial motion sensor. Then, the fourth parameter may be a value of thefourth measurement describing an amount of rotation of the second devicemeasured by the fourth sensor. The fourth parameter may in this case bea value of the fourth measurement describing an incremental rotation ofthe second device measured by the fourth sensor, or a value of aquaternion of the fourth measurement.

According to a second aspect, an apparatus is provided. The apparatuscomprises a processor configured to obtain a first device signaturecomprising an indication of a first point in time and a first parameterassociated with the first point in time, wherein the first parameter ischaracteristic of a first measurement performed by a first sensorcomprised in a first device. The processor is further configured toobtain a second device signature comprising an indication of a secondpoint in time and a second parameter associated with the second point intime, wherein the second parameter is characteristic of a secondmeasurement performed by a second sensor comprised in a second device.The processor comprised in the apparatus is further configured todetermine, based on the first device signature and the second devicesignature, a measure of proximity between the first device and thesecond device.

The apparatus may further comprise a communication interface coupled tothe processor and configured to obtain at least one device signatureselected from the first device signature and the second devicesignature.

According to a third aspect, a device is provided which comprises afirst sensor configured to perform a first measurement and generatefirst measurement data based on the first measurement. The devicefurther comprises a processor, the processor being configured to obtainthe first measurement data from the first sensor, and to determine,based on the first measurement data, a first parameter associated with afirst point in time, wherein the first parameter is characteristic ofthe first measurement. The processor comprised in the device is furtherconfigured to generate a first device signature comprising an indicationof the first point in time and the first parameter.

The processor comprised in the device in one example is furtherconfigured to determine a plurality of parameters based on themeasurement data and to select at least one of the plurality ofparameters as the first parameter.

The processor comprised in the device may be configured to implement anartificial neural network which is adapted to determine the firstparameter. The artificial neural network may be adapted to perform atleast one operation chosen from the determination of the plurality ofparameters and the selection of the at least one parameter.

In one variant, the device further comprises an output interfaceconfigured to provide the first device signature to an apparatus, forexample to the apparatus according to the second aspect.

According to one example, the device further comprises a third sensorconfigured to perform a third measurement and generate third measurementdata based on the third measurement. In this case, the processorcomprised in the device may be configured to obtain the thirdmeasurement data from the third sensor and to determine, based on thethird measurement data, a third parameter associated with a third pointin time, wherein the third parameter is characteristic of themeasurement by the third sensor. This processor may be furtherconfigured to generate the first device signature comprising anindication of the first point in time, the first parameter, anindication of the third point in time and the third parameter.

According to a fourth aspect, a computer program is provided. Thecomputer program comprises instructions which, when the program isexecuted by a processor, cause the processor to carry out the method ofthe first aspect.

According to a fifth aspect, a computer-readable medium is provided. Themedium comprises the computer program according to the fourth aspect.

According to a sixth aspect, a data carrier signal is provided. The datasignal carries information which represents the computer programaccording to the fourth aspect.

The term “parameter” used herein may also be referred to as “feature”which is characteristic of the respective measurement. It is to beunderstood that a referral to (e.g., an explanation in relation to) thefirst parameter may equally apply to the second parameter, the thirdparameter, the fourth parameter and vice versa, as far as applicable. Incase it is referred to the first sensor, it is noted that similartechnical teachings may apply to the second sensor, the third sensor andthe fourth sensor and vice versa, as far as applicable. In case it isreferred to a “parameter” in general, one or more of the aforementionedparameters is meant, as far as applicable. The same holds true for areferral to a “measurement” and a referral to a “sensor”.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details, advantages and aspects of the present disclosure willbecome apparent from the following embodiments taken in conjunction withthe drawings, wherein:

FIG. 1 shows exemplary logistical transport ways of different parcels;

FIG. 2 illustrates a network system embodiment according to the presentdisclosure;

FIG. 3 illustrates a further network system embodiment according to thepresent disclosure that may be based on the embodiment of FIG. 2;

FIG. 4 illustrates a method embodiment according to the presentdisclosure;

FIG. 5 illustrates a further method embodiment according to the presentdisclosure;

FIG. 6 illustrates an exemplary first measurement of a vibration sensor;

FIG. 7 illustrates exemplary first and second measurements ofacceleration sensors;

FIG. 8 illustrates exemplary first and second measurements of gamerotation sensors;

FIG. 9 illustrates two exemplary measurements of microphones and asuperposition of these measurements;

FIG. 10 illustrates an exemplary embodiment of an neural networkaccording to the present disclosure;

FIG. 11 illustrates experimental results of sound similarity predictionusing an neural network;

FIG. 12 illustrates experimental results based on sound and rotationmeasurements of two devices according to the present disclosure;

FIG. 13 illustrates three exemplary panoramic images acquired atdifferent locations;

FIG. 14 illustrates colour histograms of the panoramic images of FIG.13; and

FIG. 15 illustrates two exemplary colour histograms.

DETAILED DESCRIPTION

In the following description, exemplary embodiments of a surgicalnavigation system is and a surgical navigation method will be explainedwith reference to the drawings. The same reference numerals will be usedto denote the same or similar structural features.

FIG. 1 shows exemplary logistical transport ways for different parcels#1 to #3 that may benefit from the technique presented herein. As can beseen, parcels may be transported and stored individually or together. Inparticular parcels #2 and #3 are transported together on the samepallet, parcels #1 to #3 are stored together in the same storagefacility, but partially on different shelves #A and #B. In order todetermine, in certain embodiments, a relative position between two ormore of the parcels #1 to #3, the present disclosure provides atechnique as will be explained in the following. It will be apparentthat the present disclosure can be practice in many other contexts aswell.

FIG. 2 shows a network system 2 comprising a first device 4, a seconddevice 6 and an apparatus 8. The first device 4 is a mobile device andmay be attached to or contained in a parcel such as parcel #1 of FIG. 1.The second device 6 may also be a mobile device (e.g., attached to aparticular pallet in FIG. 1). In the shown example, the second device 6is a stationary device with a known absolute position (e.g., astationary device in the storage facility shown in FIG. 1 or attached toa particular shelf in FIG. 1).

The first device 4 comprises a first sensor 10 and the second device 6comprises a second sensor 12. In addition, in some embodiments, thefirst device 4 comprises an optional third sensor 14 and the seconddevice 6 comprises an optional fourth sensor 16. The first sensor 10 andthe second sensor 12 are each configured to measure a same physicalproperty. In addition, the third sensor 14 and the fourth sensor 16 areeach configured to measure a same physical property. The measuredphysical property is for example vibration, sound, light, acceleration,rotation, a magnetic field or temperature. It should be noted that thefirst device 4 and the second device 6 may also include additionalsensors.

The first sensor 10 is configured to perform a first measurement andgenerate first measurement data based on the first measurement, thesecond sensor 12 is configured to perform a second measurement andgenerate second measurement data based on the second measurement.Accordingly, the third sensor 14 is configured to perform a thirdmeasurement and generate third measurement data based on the thirdmeasurement and the fourth sensor 16 is configured to perform a fourthmeasurement and generate fourth measurement data based on the fourthmeasurement.

The first device 4 comprises a processor 18 which is configured toobtain the first measurement data from the first sensor 10 and the thirdmeasurement data from the third sensor 14. The data may then be storedin a data storage 20 comprised in the first device 4 or externally. Theprocessor 18 is further configured to determine, based on the firstmeasurement data, a first parameter associated with a first point intime, wherein the first parameter is characteristic of the firstmeasurement. For example, in case of an audio signal measured as thefirst measurement (i.e., in case the first sensor 10 is a microphone),the processor 18 may extract a largest peak value of the audio signal asthe first parameter. This peak value is associated with a certain pointin time at which the peak occurred. This certain point in time is set asthe first point in time. Similarly, the processor 18 is configured todetermine, based on the third measurement data, a third parameterassociated with a third point in time.

The processor 18 is configured to generate a first device signaturecomprising an indication of the first point in time and the firstparameter. That is, the first device signature in the given examplecontains an absolute value of the extracted largest peak of the audiosignal and the certain point in time at which the peak occurred at thefirst sensor 10 of the first device 4. The first device signaturefurther comprises an indication of the third point in time and the thirdparameter.

Also, the first device signature comprises the following data elements(or a subset thereof): an indication of an amount of parameterscomprised in the first signature, an indication of the sensor type ofthe first sensor 10, an indication of a point in time at which the firstmeasurement was started by the first sensor 10, an indication of atimespan between the point in time at which the first measurement wasstarted by the first sensor 10 and the first point in time, anindication of the parameter type of the first parameter, an indicationof the sensor type of the third sensor 14 (if present), an indication ofa point in time at which the third measurement was started by the thirdsensor 14, an indication of a timespan between the point in time atwhich the third measurement was started by the third sensor 14 and thethird point in time, and an indication of the parameter type of thethird parameter. The sensor type is indicative of the physical parametermeasured by a particular sensor, such as the first sensor 10.

The first device signature may be stored in the data storage 20. Anoutput interface 22 comprised in the first device 4 is configured toprovide the first device signature to a communication interface 24comprised in the apparatus 8. For example, the first device signature istransmitted by the output interface 22. In another example, the outputinterface 22 is connected to the data storage 20 and enables obtainingthe first device signature from the data storage 20 via the outputinterface 22 by the apparatus 8. The amount of data transferred from thefirst device 4 to the apparatus 8 is smaller compared with atransmission of the complete first measurement data. In particular, thefirst parameter consists of an amount of information that is lower thanthe amount of information in the first measurement. For that reason, itcan also be referred to an “extraction” of the first parameter from thefirst measurement. The same applies to the third parameter and the thirdmeasurement.

A corresponding configuration is provided in the second device 6.Namely, the second device 6 comprises a processor 26 and a data storageunit 28 as well as an output interface 30. The processor 26 isconfigured to obtain the second measurement data from the second sensor12 and determine, based on the second measurement data, a secondparameter associated with a second point in time. The second parameteris characteristic of the second measurement. Also, the processor 26 isconfigured to obtain fourth measurement data from the fourth sensor 16and determine, based on the fourth measurement data, a fourth parameterassociated with a fourth point in time. The fourth parameter ischaracteristic of the fourth measurement. As described above withreference to the first and second parameters, the third and fourthparameters each consist of an amount of information lower than the totalamount of information of the third and fourth measurements.

The first and second parameters are of a same parameter type, and thethird and fourth parameters are of a same parameter type. The parametertype is one of a maximum value, a minimum value, a value above apredetermined threshold or a value below a predetermined threshold. Thevalue is a value of the measurement, a derivative of the measurementwith respect to time, a Fourier-transform of the measurement, aquaternion of the measurement, a Mel spectrum of the measurement, ahistogram of the measurement, or a wavelet-transform of the measurement.The first and the second parameter each can be expressed in a vectorformat. That is, the first parameter may be referred to as a firstfeature vector whereas the second parameter may be referred to as asecond feature vector.

The processor 26 is further configured to generate a second devicesignature comprising an indication of the second point in time and thesecond parameter as well as an indication of the fourth point in timeand the fourth parameter. The second device signature further comprisesthe following data items (or a subset thereof): an indication of anamount of parameters comprised in the second signature, an indication ofthe sensor type of the second sensor, an indication of a point in timeat which the second measurement was started by the second sensor, anindication of a timespan between the point in time at which the secondmeasurement was started by the second sensor and the second point intime, an indication of the parameter type of the second parameter, anindication of the sensor type of the fourth sensor (if present), anindication of a point in time at which the fourth measurement wasstarted by the fourth sensor, an indication of a timespan between thepoint in time at which the fourth measurement was started by the fourthsensor and the fourth point in time, and an indication of the parametertype of the fourth parameter.

The second device signature may be stored in the data storage 28. Theoutput interface 30 is configured to provide the second device signatureto the communication interface 24 of the apparatus 8. The outputinterface 30 may also provide the second device signature to the firstdevice 4 via the output interface 22. The output interface 22 mayprovide the first device signature to the second device 6 via the outputinterface 30. Different possible routes for transmitting one or both ofthe device signatures are indicated with dotted lines in FIG. 2.

The first device 4 and the second device 6 can be realized as Internetof things, IoT, devices so that the output interfaces 22 and 30 are ableto connect to a network such as the Internet. The apparatus 8 may thenbe a cloud server which is connected to the same network via thecommunication interface 24. The first device 4 and the second device 6may be configured to communicate via a wireless communicationtechnology, such as the technology defined by 5^(th) Generation (5G)communication standards.

The apparatus 8 comprises a processor 32 which is configured to obtainthe first device signature from the first device 4 and the second devicesignature from the second device 6 via the communication interface 24.Alternatively, the apparatus 8 may obtain both device signatures fromonly one of the first device 4 and the second device 6. The first devicesignature and the second device signature may then be stored in a datastorage 34 comprised in the apparatus 8.

The processor 32 is configured to determine, based on the first devicesignature and the second device signature, a measure of proximitybetween the first device 4 and the second device 6. This determinationcan be performed by an artificial neural network, ANN. The determinationcomprises correlating the first device signature and the second devicesignature. In particular, the correlating comprises comparing the firstpoint in time with the second point in time and comparing the firstparameter with the second parameter. The correlating further comprisescomparing the third point in time with the fourth point in time andcomparing the third parameter with the fourth parameter.

For instance, the processor 32 is configured to determine whether thefirst device 4 is located in a same room as the second device 6. Asanother example, the processor 32 is configured to determine whether thefirst device 4 is located in a same shipment container as the seconddevice 6. In other words, the determined measure of proximitycorresponds to a relative location of the first device 4 with respect tothe second device 6. As the second device 6 in the shown example has aknown absolute position, an absolute position of the first device 4 canalso be determined based on the determined measure of proximity. Forinstance, if it is determined that the first device 4 and the seconddevice 6 are located within a same room, and the absolute position ofthe room is known due to the known absolute location of the seconddevice 6, it can be determined that the first device 4 has the sameabsolute position as the second device 6, namely the absolute positionof the same room. In other words, the apparatus 8 is configured tocalculate, as the measure of proximity, a hit score for each potentialdevice pair. In the case of only two devices such as the first device 4and the second device 6, only one hit score is determined. The betterthe match of the device signatures, the higher is the hit score and thehigher is the probability that these devices are in close proximity toone another.

A signal emitting unit may be provided at a known location such as inthe back of a truck, in a shelf of a storage facility or in a shipmentcontainer. The signal emitting unit can emit a known physical signalsuch as sound, a light pattern, a magnetic field, a vibration or else.The known physical signal may then be used to determine a devicesignature. For example, the second device 6 is configured as the signalemitting unit and is configured to emit the known physical signal. Theprocessor 26 can then be configured to determine the second devicesignature based on the known physical signal. No sensors need to beincluded in the second device 6 in this case as the second parameter canbe determined based on the known physical signal. Also, as the locationof the signal emitting unit, i.e., the second device 6, is known, amagnitude, size or time of a signal measured by the sensor(s) of thefirst device, which is configured to measure the physical signal, isinfluenced by the distance between the first device 4 and the signalemitting unit and possibly also by the relative orientations between thesignal emitting unit and the first device 4. This enables thedetermination of a distance and a relative position between the firstdevice 4 and the second device 6 based on the device signatures.

FIG. 3 shows a different schematic illustration of the system depictedin FIG. 2. In particular, individual processing steps and dedicatedcomponents are shown in FIG. 3 with reference to the individualcomponents on which the processing occurs. In addition to the firstsensor 10 and the third sensor 14, a plurality of further sensors 36,38, 40 and 42 are comprised in the first device 4. Also, in addition tothe second sensor 12 and the fourth sensor 16, several further sensors44, 46, 48 and 50 are comprised in the second device 6. Of course, insome variants the first sensor 10, the second sensor 12, the thirdsensor 14 and the fourth sensor 16 may be realized by one of the sensors36-42 and 44-50 shown in FIG. 3, for example light sensors or vibrationsensors.

As can be seen, the determination of the first and third parametersbased on individual measurement signals of the sensors 10, 14, 36, 38,40 and 42 is performed by the processor 18 of the first device 4. Thefirst or third parameter may be determined based on measurement signalsof more than one sensor. For example, a parameter may be determinedbased on signals of several acceleration sensors, as will be describedbelow with reference to FIG. 8. As another example, a peak detected inboth an audio signal measured by the microphone 14 and a vibrationsignal measured by the vibration sensor 42 may be used as the firstparameter.

The first and third parameters are determined wavelet-based, meaningbased on a wavelet transformation applied onto the measurement signals.The first and third parameters can be determined Fourier-based (i.e.,based on a frequency histogram) or by detecting peaks. Subsequently, thefirst device signature is generated by the processor 18 and transmittedvia the output interface 22 to a wireless network 52. The first devicesignature is then transferred to a signature correlation server 54 whichcorresponds to the apparatus 8 shown in FIG. 1.

Also, the determination of the second and fourth parameters is performedby the processor 26 of the second device 6 based on measurement dataobtained from the sensors 12, 16, 44, 46, 48 and 50. The second andfourth parameters are determined wavelet-based, Fourier-based (i.e.,based on a frequency histogram) or by peak detection. Subsequently, thesecond device signature is generated by the processor 26. The seconddevice signature is transmitted via the output interface 30 to thewireless network 52, from where it is transferred to the signaturecorrelation server 54. The signature correlation server 54 may be acloud server and corresponds to the apparatus 8. The signaturecorrelation server 54 is configured to perform processing of the firstand the second device signature and to determine the measure ofproximity between the first device 4 and the second device 6 based onthe first and the second device signatures.

Note that the processors 18, 26 may be configured to determine aplurality of parameters based on the available measurements andsubsequently select the most relevant of these parameters as the first,second, third or fourth parameter. For this selection, an artificialneural network, ANN, may be implemented by the processors 18, 26. ThisANN is preferably trained to select the parameters that give the bestinsight to the actual measurement data. In particular, during training,a wide range of measurement data is presented to the ANN, together withthe correct measure of proximity. For instance, measurement dataobtained by devices which are remote from one another and measurementdata obtained by devices which are close to one another, for example inthe same room, is used to train the ANN. As a result, a high accuracy inthe selection of the parameters by the ANN can be obtained.

FIG. 4 shows a flow chart of a method executed by the apparatus 8described with reference to FIGS. 2 and 3 above.

In a first step S2, the apparatus 8 obtains the first device signature.In a second step S4, the apparatus 8 obtains the second devicesignature. In a third step S6, the apparatus 8 determines the measure ofproximity between the first device and the second device based on thefirst device signature and the second device signature, as generallyexplained herein.

Note that the first step S2 and the second step S4 may be performed atthe same time or in a reverse order. Obtaining the first devicesignature and obtaining the second device signature may comprisereceiving the respective device signature, for example via outputinterfaces 22 and 30 and communication interface 24, or loadingrespective device signatures from a data storage, for example from oneor more of data storages 20, 28 and 34.

FIG. 5 shows an example of a first measurement 56 measured by the firstsensor 10 and a second measurement 58 measured by the second sensor 12.The processor 18 of the first device 4 is configured to determine thefirst parameter based on this first measurement 56. In the shownexample, the first measurement 56 contains a plurality of characteristicparameters. These characteristic parameters are denoted with referencesigns 60 to 74. On the other hand, the processor 26 of the second device6 is configured to determine the second characteristic parameter basedon the second measurement 58. Characteristic parameters of the secondmeasurement 58 are denoted with reference signs 76 to 84.

The first device signature in one example contains the parameter 68 asthe first parameter associated with t3 as the first point in time. Thesecond device signature contains the parameter 78 as the secondparameter associated with t3 as the second point in time. As notedabove, the first device signature further contains an indication of apoint in time at which the first measurement was started by the firstsensor 10 and an indication of the timespan between the point in time atwhich the first measurement was started by the first sensor 10 and thefirst point in time. In other words, the first device signature containsan indication of t1 and the timespan between t1 and t3. The seconddevice signature also comprises such information, in particular anindication of t2 and the timespan between t2 and t3.

Due to this time information, the processor 32 comprised in theapparatus 8 is able to perform a time-matching between the firstparameter and the second parameter. As indicated in FIG. 5, it ispossible to compare the first point in time with the second point intime. Here, the first point in time t3 of the first parameter 68corresponds to the second point in time t3 of the second parameter 78.In other words, the first point in time matches the second point intime. The same applies to points in time associated with parameters 70and 80, parameter 72 and 82, and parameters 74 and 84, respectively. Indifference thereto, points in time associated with parameters 60, 62,64, 66 and 76 do not match with parameters obtained from the othermeasurement signal. In addition to comparing the first point in timewith the second point in time, the processor 32 can compare the firstparameter with the second parameter.

For example, the first measurement 56 and the second measurement 58 areobtained by microphones. In this case, the individual parameters maycorrespond to peaks detected in the audio signals. Due to a differentlocation of the first device 4 with respect to the second device 6,different audio signals are obtained. For example, the first device 4may be arranged in the back of a truck in close proximity to a door,whereas the second device 6 may at the same time be arranged in the backof the truck remote from the door. Sounds generated at the door may thusonly be detected by the microphone of the first device 4 as theparameters 60, 62, 64 and 66. At the same time, larger sounds can bedetected with the microphones of the first device 4 and the seconddevice 6 as the parameters 68, 78, 70, 80, 72, 82, 74 and 84. As aplurality of time points associated with the parameters matches betweenthe first measurement 56 and the second measurement 58, it can bedetermined as the measure of proximity that first device 4 and thesecond device 6 are in a same room, or in the back of a same truck. Atthe same time, it can be determined that both devices are not in closeproximity to one another as several parameters contained in the devicesignatures do not match. In order to avoid erroneous determinationresults, the first parameter that timely matches a second parameter maybe compared therewith. In the current example, amplitude values of peaksdetected in the audio signals as the first and second parameters may becompared with one another. In case both values lie within a certainrange and are of a same type (i.e., a maximum amplitude value), thesetwo parameters are classified as matching parameters and the result ofthe previous time-matching is approved. As a result, it is determined asthe measure of proximity that the first device 4 and the second device 6are in close proximity to one another.

The processor 18 may be configured to determine a plurality ofparameters based on the first measurement data and to select at leastone of the plurality of parameters as the first parameter. The processor18 for example implements ANN adapted to determine the first parameter.One example of such an ANN is a convolutional neural network, CNN, thatis configured to determine a plurality of parameters based on the firstmeasurement using parameter detection algorithms. In other words, theprocessor 18 is configured to extract a characteristic property or acharacteristic feature of the first measurement as the first parameter.The first parameter is later compared with a second parameter of thesame parameter type in order to correlate different device signatures.

FIG. 6 shows an exemplary first measurement obtained by the first sensor10 of the first device 4. In this case, the first sensor 10 is avibration sensor. The abscissa indicates time whilst the ordinateindicates frequency. Different shadings indicate different amplitudes.In the shown diagram, regions with high amplitudes can be identified ascharacteristic of the first measurement. These regions are indicated bycircles in FIG. 6 and may each or all be used as the first parameter.The time and frequency of each of the regions can be described bytwo-component vectors denoted as [T1, F1], [T2, F2] and [T3, F3],respectively. In the shown example, these two-component vectors describea time and frequency of a maximal amplitude which lies in a regionindicated with a circle in FIG. 6. One or more of the frequencies F1, F2and F3 may be used as the first parameter associated with the point intime T1, T2 or T3, respectively. Alternatively, vectors with a higherdimension describing the circles shown in FIG. 6 or a range of time anda range of frequency may be used. Each vector describes a parametercharacteristic of the first measurement and a point in time associatedwith the parameter. Instead of detecting maximal amplitudes, patterns inthe time-frequency diagram may be detected as the parameterscharacteristic of the first measurement. For this purpose, the processor18 may implement an ANN.

In FIG. 7, a first measurement of the first sensor 10 and a secondmeasurement of the second sensor 12 are shown. In this case, the firstsensor 10 and the second sensor 12 are acceleration sensors. In theshown example, the first device 4 and the second device 6 were placed ina common parcel which was then moved by hand.

It can be seen that the first measurement is highly similar to thesecond measurement. In particular, time points at which the firstmeasurement shows maxima and minima are closely similar to time pointsat which the second measurement shows maxima and minima. In case amaximum of the first measurement signal at a first point in time (forexample at 0.95 s) is determined as the first parameter, and a maximumof the second measurement signal at a second point in time (for exampleat 0.96 s) is determined as the second parameter, the apparatus 8 cancompare the first point in time with the second point in time based onthe obtained device signatures. In the example, a deviation betweenthese points in time is derived during time-matching of the first devicesignature and the second device signature as 0.96 s-0.95 s=0.01 s. Theapparatus 8 determines that, due to the same type of parameter and dueto the closely similar points in time with which the parameters areassociated, the first device 4 and the second device 6 are in closeproximity to one another as they exhibit closely similar temporalacceleration.

The absolute parameter values may also be compared to one another. Inthis example, the first parameter and the second parameter both have anacceleration value of about 1.255 g. This means that the first devicesignature and the second device signature are not only highly correlatedwith respect to the first and second points in time, but also withrespect to the values of the first and second parameter. Consequently,the apparatus 8 determines that the first device 4 and the second device6 are in close proximity to one another. In other words, a co-localityof the two devices 4 and 6 can be proven by the apparatus 8 based on thefirst and second device signatures generated from the measurements shownin FIG. 7.

The first and second parameters may each be determined based on morethan one measurement signal. In particular, according to a more advancedinertial measurement unit, IMU, based method, a short time scalerotation of the first device 4 or the second device 6 is sampled basedon sensor fusion, i.e. based on superimposed measurement signals ofseveral sensors. A commonly available 6-axis fusion of an accelerometerand a gyroscope, usually called game sensor or game rotation sensor, maybe used in order to determine the first parameter or the secondparameter. Even if the first device 4 and the second device 6 havedifferent initial rotational positions, if they are moving together,they experience a similar sequence of rotations. Using quaternions, therelation between the original position P1 and the new position P2 can beexpressed as:

P ₂ =Q ^(c) ×P ₁ ×Q,

where Q expresses the rotation between points P₁ and P₂, and Q^(C) isthe conjugate of Q. A convenient property of quaternions is thatsuccessive rotations are multiplications, i.e.: Q^(c)=K×Q₁, so the newrotation quaternion Q₂ is a multiplication of the previous rotation Q₂and the further rotation of K. If two devices are handled together,regardless of their prior rotational history, they will experiencesimilar incremental rotations. In practice, the incremental rotation Kcan be determined from a current and previously sampled normalizedquaternions:

K=Q ₂ ×Q ₁ ^(c).

The choice of sampling frequency will impact whether small timescalemotion or large motions are sought to be detected as the (i.e., first,second, third or fourth) parameter. Device signatures in which thefirst, second, third or fourth parameter is based on quaternions tend toprovide better results than device signatures in which the parametercorresponds to an absolute accelerometer measurement value. That isbecause fine motions can be detected with high precision and low noisebased on quaternions.

FIG. 8 shows measurement results of an experiment where the first device4 and the second device 6 were first transported separately (between0-15 sec) and then transported in a same container (between 15-30 sec).In this diagram, measurements obtained by a game rotation sensor areindicated as w, x, y and z. The measurements of the game rotation sensorcomprised in the first device 4 are shown in the top four diagrams witha solid line and denoted as “D1”, whereas the measurements of the gamerotation sensor comprised in the second device 6 are shown in thesediagrams with a dashed line and denoted as “D2”. A 100 ms sampling of Kwas used, which optimizes for small to medium time-scale signaturematching.

The bottom diagram shown in FIG. 8 represents an Euclidean length ofrotational quaternion vector differences, wherein the rotationalquaternion vectors are derived from the measurements of the gamerotation sensors comprised in the first device 4 and the second device6, respectively. As the first and second parameters included in thedevice signatures, maxima or minima of the measurements w, x, y and z ora quaternion vector comprising values of w, x, y and z at a certainpoint in time can be determined. Alternatively or additionally, certainpatterns may be detected in the measurements w, x, y and z as theparameters, for instance by an ANN implemented by the processor 18 ofthe first device 4 or the processor 26 of the second device 6.

It can be seen that, when the two devices 4 and 6 are carried together(between 15-30 sec), the values of measurements w, x, y and z, which arethe quaternion vector values, are very similar. Also, the Euclideanlength of the rotational quaternion vector differences shown in thebottom diagram is significantly smaller than when the first device 4 andthe second device 6 are carried separately by two persons (between 0-15s). This enables the determination by the apparatus 8 that during 0-15s, the first device 4 and the second device 6 are transported separatelyfrom one another, whilst during 15-30 s, the first device 4 and thesecond device 6 are transported together as they experience the sameincremental rotations.

A 6-axis IMU such as a game rotation sensor generally has larger powerconsumption than a regular acceleration sensor. The power consumptioncan be minimized by keeping the sensor powered down for most of thetime. This approach is feasible because only measurement samples ofincrements at a low rate are required compared to a full history ofrotations. The sensor may only be activated when motion is detected byan ultra-low power accelerometer. The first measurement may be startedby the first sensor 10 upon a signal of the third sensor 14.

As noted above, microphones may be comprised as sensors in the firstdevice 4 and the second device 6. Microphones may detect similar soundpatterns in an environment. The two devices 4 and 6 may experience asignificant amount of additional, different noise even if they are inthe same room. In order to obtain useful information about the locationof the first device 4 and the second device 6 based on measurements ofmicrophones as the first sensor 10 and the second sensor 12, theparameters characteristic of the measurements are determined by theprocessors 18 and 26.

In order to determine whether the first device 4 and the second device 6are in close proximity to one another, a decision function can be used.In the following decision function f, A₁ denotes the first measurementobtained by a microphone as the first sensor 10 and A₂ denotes thesecond measurement obtained by a microphone as the second sensor 12.Each of these measurements can be considered as a sum of a sound patternC with some attenuation α and β (potentially time dependent) while allthe other sound in the measurement considered as background sounds isdenoted by N₁ and N₂:

A ₁ =α×C+N ₁;

A ₂ =β×C+N ₂.

Since the sound pattern C can be very weak and the background noise N₁and N₂ can be strong compared to the sound pattern C, the followingconditions hold:

α,β<<1

α×C<<N1

β×C<<N2.

The processor 32 of the apparatus 8 is configured to use the decisionfunction f that enables a decision whether the two measurements A₁ andA₂ contain the same sound pattern C:

${f\left( {A_{1},A_{2}} \right)} = \left\{ \begin{matrix}1 & {{{{if}\alpha} \neq 0},{\beta \neq 0}} \\0 & {otherwise}\end{matrix} \right.$

To demonstrate the task of finding such a function f, Mel spectralanalysis can be used in order to compress a Mel spectrogram of each ofthe measurements A₁ and A₂, respectively.

FIG. 9 shows exemplary measurements A₁, A₂ and a weighted superpositionB of these measurements in the bottom row, wherein the superposition Bconsists of 80% of the intensity of A₁ and 20% of the intensity of A₂.The measurements A₁ and A₂ in the provided example were taken with a 22kHz raw sample rate, and a Fast Fourier Transform, FFT, window size of4096. For each of the measurements A₁ and A₂ as well as forsuperposition B, corresponding compressed Mel spectrograms are shown inFIG. 9 in the top row. In this example, the Mel spectrograms werecompressed to a 32×32 matrix.

It can be seen that while the original waveform of the measurement A₂ isalmost not visible in the waveform of superposition B, the specificpatterns of both the spectrograms of A₁ and A₂ can be recognized inspectrogram of B. In case the weighting of measurement A₂ is decreasedin superposition B, the recognition of the measurement A₂ in thewaveform or the compressed Mel spectrogram of B gets impossible forhuman senses.

The parameter extraction and signature generation should be performed bythe devices 4 and 6, while the determination of the measure of proximityshould take place in a cloud component such as the signature correlationserver 54. To prove feasibility of the use of an ANN for parameterextraction and signature generation and also use of an ANN fordetermining the measure of proximity, in the following, a machinelearning solution is presented that includes all of the determination ofthe characteristic parameter, the determination of the device signaturesand the determination of the final decision, i.e., of the measure ofproximity.

FIG. 10 shows an exemplary structure of a Siamese ANN. The Siamese ANNcontains two separated input branches and a common part. The separatedinput branches illustrate the devices' parameter extraction andsignature determination and the common part embodies the cloud'ssignature processing and determination of the measure of proximity. Theleft branch of the Siamese ANN may be implemented as an ANN by theprocessor 18 of the first device 4. The right branch of the Siamese ANNmay be implemented as an ANN by the processor 26 of the second device 6.The common layer may be implemented by the processor 32 of the apparatus8. The Siamese ANN shown in FIG. 10 is designed to find similaritiesbetween two recorded sound sequences, i.e., between the firstmeasurement A₁ and the second measurement A₂. For testing thesimilarity, recorded sound sequences were in an experiment split into5.8 sec long sequences and their distinct spectrograms were calculated.The spectrograms contain a correlation in both the time and frequencydomains of the recorded sound sequences.

The correlations between the spectrograms of the individual sequencescan be extracted effectively with convolutional layers that areorganized as follows (see FIG. 10): the two input branches contain thesame layers in the same order: a Convolutional (16,3,3), a Convolutional(16,3,3), a Maxpooling (4,4) to reduce dimensions of the activation mapsand the number of parameters, a Convolutional (16,3,3) and a Maxpooling(4,4) layer. The two branches are attached with a fully connected layerand the network's output layer contains two neurons with Softmaxactivation function.

During a training process of the Siamese ANN, in the experiment, a largenumber of sound records (of 5.8 sec length) from various environmentswere used and an audio mixing process was applied as a data augmentationstep. In the mixing process, a random sound record R₁ was used as one ofthe inputs while a mixed record A_(m) was used as the other input. Themixed record A_(m) was calculated as a weighted average of the soundrecord R₁ and a randomly selected sound record R₂. By using this mixingprocess, the amount of the available training data was increased. Also,training data pairs were generated that contained adjustable fractionsof R_(i) beside R₂. An example for such a mixing process was mentionedabove with reference to FIG. 8, in which B corresponds to A_(m), A₁corresponds to R₂ and A₂ corresponds to R₁. According to the mixingprocess, the sound recordings A_(m) are calculated as follows:

A _(m) =α×R ₁ +β×R ₂.

For 0.2<α<1, 0<β<1, the input record pairs R₁ and R₂ were considered asmatching (i.e., true) sound patterns, which should lead to adetermination that the first device 4 and the second device 6 are inclose proximity to one another or in a same room. For α=0, the selectedpairs were considered as non-matching (i.e., false or independent) soundpatterns, which should lead to a determination that the first device 4and the second device 6 are not in close proximity to one another or notin a same room.

The training dataset finally contained 64.000 record pairs. This numberwas doubled (to 128.000 records) due to a swapping of pairs to ensure anidentical training of the two branches of the Siamese ANN shown in FIG.10. The dataset was split into 80% used for training of the Siamese ANNand 20% used for validating the Siamese ANN. The neural network wasimplemented in Keras. A Nesterov-Adam optimizer was used with 0.001learning rate. As loss function, a categorical cross entropy was used.Early stopping was applied to avoid overtraining, after 40 epochs theANN reached 0.7374 accuracy and 0.7954 validation accuracy. Thedifference between training and validation accuracy is due to a useddropout layer that deactivates neurons during training of the ANN butactivates them during validation of the ANN. This dropout layer waslocated between the two fully connected layers and is used to avoidoverfitting.

The validation results of the Siamese ANN of FIG. 10 are shown in FIG.11. The mixing ratio in this figure means the weighting of soundrecording R₁ that was used during mixing to generate the differentsuperimposed recordings A_(m). The accuracy in this figure means thepercentage of correct decisions determined by the Siamese ANN. Inparticular, in case the Siamese ANN determined that the first device 4and the second device 6 are in close proximity to one another based ontwo superimposed patterns A_(m), whilst these patterns A_(m) did notcontain the same pattern R₁, this determination was incorrect. On theother hand, in case the Siamese ANN determined that the first device 4and the second device 6 are in close proximity to one another based ontwo superimposed patterns A_(m), whilst these patterns A_(m) did containthe same pattern R₁, this determination was correct. As can be seen, thehigher the mixing ratio, the higher the accuracy. This is because theinfluence of the randomly selected R₂ decreases with higher mixingratios, enabling a more accurate decision by the Siamese ANN based ontwo different superimposed recordings A_(m). Further experiments withdifferent record lengths showed that for records containing highlysimilar sound patterns (above 80% mixing ratio), the accuracy is above80% even for short (5.8 sec) records, while it was above 95% for longer(11.6 and 17.4 sec) records. For low mixing ratios (below 20%) a longerrecord length has clear advantage as it can still achieve a highaccuracy.

Performing audio sampling, analog-digital, AD, conversion and FastFourier Transformation, FFT, of the measurements measured by the sensorscomprised in the first and second device 4 and 6 would require muchpower which is disadvantageous for a battery-based device 4 or 6. Inorder to save power, very low power MEMS sensors with very low powerwake-up capability may be used. For example, a sensor using a so-calledzero power operation may be used, which means that the detection worksby utilizing the energy harvested from the sound wave itself. The sensorshould sensitive to a relatively broad range of frequencies, for example300 Hz-6 kHz. Preferably, the sound pressure level of the sensor isadjustable. In this case, while searching for sound, the device 4 or 6requires a mere 10 uW for operation. Once the sensor detects sound, itcan wake the rest of the device 4 or 6 up, and the required sampling andFFT can be triggered.

A real-time experiment was performed. In the experiment, two parcelswere used, a first parcel containing the first device 4 and a secondparcel containing the second device 6. The parcels were handledseparately first, then they were carried together with different meansof transport, while the sound recordings of the first sensor 12 and thesecond sensor 14, and measurement data of the game rotation sensorscomprised in the devices were continuously collected.

The results of this experiment are presented in FIG. 12. The soundsimilarity prediction is shown in the top row, the analyzed rotationquaternion-based difference in the middle row, while the bottom rowshows the actual scenario. The sound similarity prediction values shownin FIG. 12 were obtained using the ANN described above with reference toFIGS. 9 to 11. The quaternion based difference is the Euclidean lengthof the rotational quaternion vector differences which was obtained asdescribed with reference to FIG. 8.

In the first scenario a), the parcels were carried separately atdifferent locations, then the parcels were placed for a while in thesame room with no movement (see scenario b)), then the parcels weretransported with a truck as shown in scenario c). In scenario d), ajoint movement was performed by the parcels by carrying the parcelstogether by one person. In scenario e), two separate persons carried thetwo parcels at different locations.

For scenarios where the two parcels are co-located, the sound similarityprediction shows high prediction values (see scenarios b), c) and d)),while the opposite can be seen in scenarios a) and e), where the parcelswere at different locations.

By investigating the quaternion-based differences one might see clearevidence of the parcels moving together (see scenarios b), c) and d)),due to the low difference values. On the other hand, the difference ishigh when the movement of the parcels is independent from each other ascan be seen in scenarios a) and e).

Neither sound-based nor quaternion-based determination of the measure ofproximity performed equally well every time. When there was no movementfor example (see scenario b)), a sound-based determination can bereliably used. On the other hand, a sound-based determination does notprovide high prediction values every time, only when there is asufficient amount of common sound signals recorded by both the firstsensor 10 and the second sensor 12. By combining independent sources,the confidence in successful co-location detection can be increased. Inother words, the determination of the measure of proximity can be basedon a first device signature which comprises not only the firstparameter, but also the third parameter and based on a second devicesignature which comprises not only the second parameter, but also thefourth parameter. In this case, the first and the second sensor 10, 12measure the same physical property, and the third and fourth sensor 14,16 measure the same physical property which is different therefrom.Measurements of sensors measuring different physical properties yield aplurality of parameters which are included in the device signatures andimprove the accuracy or reliability of the determination of the measureof proximity.

As a next example, the determination of the first parameter based on ameasurement of an image sensor (i.e., a camera) as the first sensor 10is given. A cheap, small resolution imaging sensor can be used as thefirst sensor 10 to sample surroundings of the first device 4. Note thatthe content of the image itself is not so important, such that lowperformance and quality requirements apply to optics and the imagesensor. A possible solution is to apply a low quality 360-degree opticson top of a low-resolution image sensor. The captured image data can bedirectly converted to a RGB or HSV colorspace by the processor 18 or amicrocontroller configured to perform image processing. As will be laidout below, the color histogram is used to determine the first parameter.The first device 4 does not to store the image, as the color histogramcan be calculated on-the-fly during data read-out by the processor 18 orthe microcontroller attached to the first sensor 10.

Features (i.e., parameters) characteristic of the measurement of theimage sensor are extracted as the positions of dips and peaks in thecolor histogram. The extracted features may be analyzed by a local AIcomponent such as an ANN implemented by the processor 18 of the firstdevice 4, in order to recognize if the captured scene has changed, i.e.whether the first device 4 has been moved or not. The ANN is used toextract such a feature based on the color histogram, for example theweight of various color ranges, or the ratio or their weight withrespect to each other.

Another AI component, such as an ANN, is implemented by the processor 32of the apparatus 8 to detect if the first device 4 and the second device6 are sharing the same location, since their color histograms do notdiffer significantly from each other. Here, as the another AI component,one might use two or more fully connected hidden layers which have beentrained to check if two local AI components extracted similar features.Such hidden layers are for example described in “Artificial Intelligencefor Humans, Volume 3: Deep Learning and Neural Networks” by Jeff Heaton(ISBN 1505714346).

In particular, the apparatus 8 is configured to determine the measure ofproximity based on the first and second device signatures which containa first and second parameter, respectively, wherein the first and thesecond parameter have been determined based on a color histogram. Ofcourse, each of the first device 4 and the second device 6 needs tocomprise at least one image sensor in order to determine theaforementioned parameters based on the respective color histogram.

FIG. 13 shows three exemplary images acquired in different locations.The top two images denoted “room #1” and “room #2” were acquired atdifferent locations in a same room, whilst the bottom image denoted“corridor” was acquired in a corridor separate from the room.

FIG. 14 shows color histogram curves generated based on the images shownin FIG. 13. The histograms of the images acquired in the same room areclosely similar to one another, whilst the histogram of the imageacquired in the corridor deviates therefrom significantly. In case thefirst parameter is extracted from the color histogram of the image “room#1” as a local maximum frequency at a hue value of ˜212 and the secondparameter is extracted from the color histogram of the image “room #2”as a local maximum frequency at a hue value of ˜214, the apparatus 8 candetermine that the first device 4 and the second device 6 are in thesame room.

In case of the image acquired in the corridor, the first parameter maydescribe a local maximum frequency at a hue value of ˜250 whilst thesecond parameter—as above—describes a local maximum frequency at a huevalue of ˜214. The apparatus 8 can then determine that the hue values ofthe first parameter and the second parameter are not closely similar,and that thus, the first device 4 and the second device 6 are notlocated within the same room.

Time correlation may also be performed by comparing the time points atwhich the images were acquired by the image sensors of the first andsecond device 4 and 6. In case the images were acquired within apredetermined time interval, for example within 1 minute, the apparatus8 determines that due to the matched hue values of the first parameterand the second parameter, which are of the same parameter type, thefirst device 4 and the second device 6 are in the same room. In case ofimage acquired a longer time apart from each other, i.e., not within thepredetermined time interval, the apparatus 8 may abort the determinationof the measure of proximity or obtain one or more new device signatures.The apparatus 8 may send an instruction to one or both of the firstdevice 4 and the second device 6 to generate a new device signature. Forthe determination of the measure of proximity, the frequency value atthe local maximum frequency may also be comprised in the first parameterand the second parameter and compared with one another. Instead of localor global maxima or minima, certain patterns may be detected as thefirst and second parameters based on the color histogram. To this end,an ANN implemented by the processors 18, 26 can be used.

FIG. 15 shows two further color histograms generated from two otherimages. It can be seen that peaks are present in each histogram whichmay be extracted as the parameters characteristic of the images. In theshown example, peaks at hue values of ˜30-45 are present in bothhistograms which would lead to a positive matching, i.e., a highcorrelation, of the first parameter with the second parameter, and thusto the determination that the first device 4 and the second device 6 arelocated in the same room or co-located.

In order to save power of the first device 4 and the second device 6comprising image sensors, very low processing capable microcontrollerssuch as ARM Cortex-M devices may be used for image processing. Thereexist ultra-low power image sensors that consume less than 2 mW powerwith a reasonable pixel resolution of 320×240. They can be duty cycledto create 1 frame every 1 minute or so, in which case the average powerconsumption drops to approximately 1.1 μW, well within the feasiblerange for battery-based operation.

As has become apparent from the exemplary embodiments, the describedtechnique herein allows a determination of the measure of proximitybetween the first device 4 and the second device 6 and, thus, a relativetracking of these two devices, even in case the devices are locatedinside a shipment container, a room, a shelf of a storage facility, aback of a truck or any other contained (e.g., walled or boxed)environment.

As “slim” device signatures of the devices 4 and 6 can be correlatedwith one another by the server 54, no complete records of sensormeasurement data needs to be transmitted from the devices 4, 6, therebysaving power of the devices 4, 6 and transmission resources. Thecorrelation of the device signatures is faster than a correlation ofcomplete sets of measurement data which enables a fast location of thedevices based on the device signatures. In case the device signatureseach comprise multiple parameters determined based on measurements fromdifferent sensor types, the determination of the measure of proximity iseven more reliable. By providing a signal emitting unit which emits aknown physical signal, an absolute position of the first device 4 may bedetermined based on the device signatures.

While the invention has been described with reference to exemplaryembodiments, it will be apparent to the skilled person that theseembodiments may be modified or supplemented in many ways. Therefore, theinvention is only limited by the claims that follow.

1-43. (canceled)
 44. A method of determining a measure of proximitybetween a first device that is mobile and a second device, the methodcomprising: obtaining a first device signature comprising an indicationof a first point in time and a first parameter associated with the firstpoint in time, wherein the first parameter is characteristic of a firstmeasurement performed by a first sensor comprised in the first device;obtaining a second device signature comprising an indication of a secondpoint in time and a second parameter associated with the second point intime, wherein the second parameter is characteristic of a secondmeasurement performed by a second sensor comprised in the second device;and determining, based on the first device signature and the seconddevice signature, the measure of proximity between the first device andthe second device.
 45. The method of claim 44, wherein the determinationof the measure of proximity is performed by an artificial neuralnetwork.
 46. The method of claim 44, wherein the first measurementconsists of a first amount of information and the first parameterconsists of an amount of information lower than the first amount, andwherein the second measurement consists of a second amount ofinformation and the second parameter consists of an amount ofinformation lower than the second amount.
 47. The method of claim 44,wherein determining the measure of proximity comprises correlating thefirst device signature and the second device signature.
 48. The methodof claim 47, wherein the correlating comprises comparing the first pointin time with the second point in time and comparing the first parameterwith the second parameter.
 49. The method of claim 47, wherein the firstdevice signature further comprises an indication of a third point intime and a third parameter associated with the third point in time,wherein the third parameter is characteristic of a third measurementperformed by a third sensor comprised in the first device; and whereinthe second device signature further comprises an indication of a fourthpoint in time and a fourth parameter associated with the fourth point intime, wherein the fourth parameter is characteristic of a fourthmeasurement performed by a fourth sensor comprised in the second device.50. The method of claim 49, wherein the correlating further comprisescomparing the third point in time with the fourth point in time, andcomparing the third parameter with the fourth parameter.
 51. The methodof claim 44, wherein the first device signature and the second devicesignature each comprise at least one further entry chosen from: anindication of the number of parameters comprised in the devicesignature; an indication of the sensor type of the sensor that performedthe measurement; an indication of a point in time at which the sensorstarted the measurement; an indication of a timespan between the pointin time at which the measurement was started by the sensor and the pointin time associated with the parameter indicated in the device signature;and an indication of the parameter type of the parameter included in thedevice signature.
 52. The method of claim 44, wherein the first sensorand the second sensor are configured to measure a same physicalproperty.
 53. The method of claim 52, wherein the physical property isone of vibration, sound, light, acceleration, rotation, magnetic fieldor temperature.
 54. The method of claim 44, wherein the first parameterand the second parameter are parameters of a same parameter type. 55.The method of claim 54, wherein the parameter type is one of a maximumvalue, a minimum value, a value above a predetermined threshold or avalue below a predetermined threshold, wherein the value is a value ofthe measurement, a derivative of the measurement with respect to time, aFourier-transform of the measurement, a quaternion of the measurement, aMel spectrum of the measurement, a histogram of the measurement, or awavelet-transform of the measurement.
 56. The method of claim 44,wherein the first sensor is a camera and the first parameter is a valueof a histogram of the first measurement by the camera, and wherein thesecond sensor is a camera in the second device and the second parameteris a value of a histogram of the second measurement by the camera in thesecond device.
 57. The method of claim 56, wherein the histogramrepresents a color distribution of pixels contained in an image acquiredby the camera as the first measurement.
 58. The method of claim 44,wherein the first sensor is a microphone and the first parameter is avalue of a Fourier-transform of the first measurement or awavelet-transform of the first measurement, and wherein the secondsensor is a microphone and the second parameter is of a same parametertype as the first parameter.
 59. The method of claim 44, wherein thefirst sensor is a microphone and the first parameter is a value of thefirst measurement, the value describing a sudden temporal change in aspectrum of the first measurement, and wherein the second sensor is amicrophone and the second parameter is of a same parameter type as thefirst parameter.
 60. The method of claim 44, wherein the first sensor isa microphone and the first parameter is a value of a Mel spectrogram ofthe first measurement, and wherein the second sensor is a microphone andthe second parameter is of a same parameter type as the first parameter.61. The method of claim 44, wherein the first sensor is a gyroscope, anacceleration sensor or an inertial motion sensor, and wherein the firstparameter is a value of the first measurement describing an amount ofrotation of the first device measured by the first sensor, and whereinthe second sensor is of a same sensor type as the first sensor and thesecond parameter is of a same parameter type as the first parameter. 62.An apparatus comprising a processor configured to: obtain a first devicesignature comprising an indication of a first point in time and a firstparameter associated with the first point in time, wherein the firstparameter is characteristic of a first measurement performed by a firstsensor comprised in a first device; obtain a second device signaturecomprising an indication of a second point in time and a secondparameter associated with the second point in time, wherein the secondparameter is characteristic of a second measurement performed by asecond sensor comprised in a second device; and determine, based on thefirst device signature and the second device signature, a measure ofproximity between the first device and the second device.
 63. A devicecomprising: a first sensor configured to perform a first measurement andgenerate first measurement data based on the first measurement; and aprocessor configured to: obtain the first measurement data from thefirst sensor; determine, based on the first measurement data, a firstparameter associated with a first point in time, wherein the firstparameter is characteristic of the first measurement; and generate afirst device signature comprising an indication of the first point intime and the first parameter.